DOCOHENT RESOHB 

ED 091 405 TH C03 614 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



EDRS PRICE 
DESCRIPTORS 



Bolton^ Dale L. 

Implications of Pupil Growth Measures for 

Administrative Action. 

[74] 

12p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (Chicago, 
Illinois, April, 1974) 

MF-$0.75 HC-$1.50 PLUS POSTAGE 

Accountability ; Achievement Gai ns ; ♦Administrator 
Guides; ♦Administrator Role ; ♦Cognitive Measurement; 
♦Decision Making ; Evaluation ; Growth Patterns ; 
Management Systems; Research Utilization; ♦Student 
Improvement ; Teacher Administrator Relationship; 
Teacher Evaluation 



ABSTRACT 

This paper relates work in the area of pupil growth 
measures to administrative tasks and decision-making. The topics 
synthesized include measurement of pupil growth, accountability and 
evaluation, and personnel systems management. A critical review, 
analysis and synthesis of pertinent literature is the major source of 
information, although information from recent experience in 
development and use of training packages for administrators is also 
used. Conclusions answer questions of adequacy of pupil growth 
measures as a basis for making administrative decisions. This should 
have implications for determining administrative roles and training 
for such roles. (Author) 
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The administrator is a peculiar person. Wo all "know'' that, but let me 



CT^ give you my impressions of his peculiarities. He has the responsibility for 

providing leadership for a group of specialists, each of whom should have more 

i — > 

^ ^ I expertise in his specialty than the administrator. Likewise, he has the 

responsibility for devising a strategy for management that will utilize the 
most current infomation regarding national commitments, local idiosyncrasies, 
curriculum development, theories of learning, and increasingly complex and 
voluminous results of studies regarding the relationships of process and 
product variables in various environments. He attempts to devise a strategy 
that will have some stability over time, knowing all the while that the infor- 
mation he has will change quite rapidly. And he should attempt to do this 
without seriously entcrtaindng paranoid thoughts that the educational psychol- 
ogists are conducting conflicting studies simply to confuse him ; therefore, he 
constantly reminds himself that his own experiential data (though not allowing 
him to attach a probability to it) leads him to believe that they are confused 
too. Small comfort. 

This paper is designed to do the following: (a) present a major theme with 
regard to the use of research information on the relationship of teaching 
processes and products, (b) provide some assumptions underlying this major theme, 
(c) undergird these assumptions with information from the research literature, 
and (d) present specific implications of these views for administrative practice. 
The Theme 

Administrators and teachers should use the results of research concerning 
presage, environmental, process, and product variables as a first estltnate of 
what is likely to result from given procedures for a specific situation. 
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Administratorw iusslst teachers in preparing for teaching activities,, in col- 
lecting infornintion rc[;arding these activities and their results, and in 
reviewing the results so that new preparations can be made. Wl^en preparations 
are being made, it is profitable to estimate the probable results of certain 
actions on the basis of inferences from prior correlational or experimental 
studies; however, though these studies are of major interest, they should not 
be consider ' as rules to be followed. Rather, they are first estimates, to be 
adjusted on the basis of review and evaluation of the results of actions chosen. 
If change is needed, a study .of other research results should assist in making a 
second estimation of what will likely work. 

This theme is based on the idea that the administrator is interested in 
helping teachers eventually to answer their question, "\7hat works (or produces 
the results I desire) for me, in my particular situation?" To answer the 
question involves trying something, documenting precisely what that something 
was, and specifying what the results were. Note that the question concerning 
the teacher is not any of the following. 

^* What am I goinj> to do? This is a process oriented question, that some 
curriculum specialists advocate, but it is often answered in terms of 
the types of activities enjoyed by the teacher or the students. The 
enjoyment may not be correlated with pupil growth measures. 
2« I^^at am I trying to accomplish ? The product oriented question has been 
avoided in the past; answering the question (or even the question of 
what was accomplished) does not allow either the diagnosis of problems 
encountered or replication of desirable results. 
3. What should do In the classroom ? When this question is answered on 
the basis of research (in a rule-following procedure) without the 
adequate follow-up of review and evaluation of results, it is likely 
to be a very sterile approach. 
Would the application of procedures implied in this theme be respectable 
from a scientific viewpoint, or am I really talking about an artistic effort or 
the practice of a craft? (I have never been enamoured with many div «ions of 
this topic before, because I am never sure of the distinctions whicli .op^^ make 
when they use these terms.) I am of the opinion that teaching involves elements 
of each of them, particularly as viewed by the following comments. 

Even if the present analyses prove not to be viable, they will not be 
replaced by the old, global, conceptually Impossible, complex variables 
that I see as one reason for the f ruitlessness of so much of research on 
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teachiiiy in the past. Instaad, they will bo replaced by other analyses of 
teachiiig, perhaps even finer analyses, until we get the sets of lawful 
relationships between variables that v;Ill mark the emergence of a scien- 
tific basis for the practice of teaching. .eventually , of course, we shall 
have to put teaching back together again into syntheses that are better than 
the teaching that goes on nov;. I think it would bo safe to say that there 
is abundant hope of our being able to develop a scientifically grounded set 
of answers to every teacher's central question, 'What should I do in the 
classroom?' (Gage, 1968, 606) 

Consider the following two views v/hich contrast with the above: 

....laws and facts, even when they are arrived at in genuinely scientific 
phape, do not yield rules of practice . Their value for educational practice — 
and all education is a mode of practice, intelligent or accidental and 
routine — is indirect; it consists in provision of intellectual instrumon- 
talitios to be used by the educator ... .That is, they direct his attention , in 
both observation and reflection, to conditions and relationships which would 
otherv;lse escape him. If we retain the word "rule" at all, we must say that 
scientific results furnish a rule for the conduct of observations and in- 
quiries , not a rule for overt action. They function not directly with 
respect to practice and its results, but indirectly, through the medium of an 
altered mental attitude. (Dewey, 1929, 28,30) 

I believe most of us who teach would admit, if pressed, that the majority of 
our teaching decisions in the classroom are not made on the basis of the- 
oretical considerations at all. Part of this is probably due to the lim- 
itations of theory in the behavioral sciences at the immediate decision- 
making level in the classroom. Human interactions are too rich to be 
adequately described by behavioral-science theory at its present state of 
development. As theory grows and as it becomes more complete, we may reach 
a point at which it may be more useful, but the likelihood of ever obtaining 
a network of theory adequate for directing all or most human action in the 
clasFroom seems to me to be small. (Eisner, 1963, 305) 

By way of an additional contrast, the following dialogue between Igor 

Stravinsky and George Balanchine is interesting: 

Stravinsky : How much music will you want for the three dancers* first 
variation? 

Balanchine : Thirty-one seconds, I would think. 
Stravinsky : Could you settle for thirty-two? 

They were not joking. One reason the late composer Igor Stravinsky and 
choreographer George Balanchine got on so well was that they both worried 
about craft at a time when everyone else was worrying about art. If art was 
the result of their labors, so much the better, but they did not agonize 
about it. "When I know how long a piece must take, then it excites me." 
Stravinsky said in explaining the importance of the discipline of limits. 
To him as to Balanchine, mastery of the work at hand was what counted, not 
the creation of so-called masterworks. As Balanchine once put it: "If you 
set out deliberately to make a masterpiece, how will you ever get it finished?" 

That masterpieces resulted anyway was amply proved last week as the New 
York City Ballet staged one of the cultural, or craftsman-like, events of the 
decade. (Time Magazine, July 3, 1972) 
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If one* viewM the coniinentK of Gage as being concerned with a science of 
teaching, those of Dov;ey and Eisner emphasirsing the artistic, and those of 
Balanchine and Stravin5;ky (by analogy) exhibiting a concern for craftsmanship, 
then I would have to think that the theme I am expressing incorporates elciuents 
of each. In essence, I would like to see the administrator assisting the 
teacher in practicing a craft, in an artistic fashion, based on scientific 
results to date. 
Assumptions 

What are the assumptions underlying this position? First , the administra*:or 
must deal with the reality of the present. He must deal with what ^L* which 
includes teachers, students, parents, expectations, motivations, learning, 
societal and environmental factors. This reality with which he must interact 
does not come packaged in the manner he might desire it, nor does it remain 
static for long periods of time; the reality he faces is dynamic, even if the 
administrator does not touch it. This is disturbing to some administrators, 
since there is a strong desire to make progress by building on the stable con- 
ditions of yesterday. The direct implication of this assumption that the admin- 
istrator must deal with the reality of the present is that he should be able to 
measure the reality that he faces accurately, expeditiously, and — if possible — 
economically. 

Second , it is assumed that there is a necessity for the administrator to 
deal with the "realm of possibility" of the future. This "realm of possibility" 
is what CAN (or' probably will) occur. Therefore, the administrator v7ho is 
interested in improvement (And v7ho could be against that?), should try to change 
those things that have a high probability of resulting in improvement; but he must 
deal with what reality is likely to be, not with dreams of what he would like for 
it to be if he were given perfect conditions. This implies that he should have 
good hunches regarding what should be changed (given certain desired outcomes), 
which variables he should try to manipulate, how people are likely to respond. 

Third , it is assumed that it is necessary for the administrator to have a 
system of management that allows him to cope with the results of his own manip- 
ulations of the environment. To cope with what happens and to make adjustments 

*Some people view life as lived on level ground, while others realize that 
it actually is lived on an inclined plane — where you must continually be moving 
upward to keep up with the traffic, and where things have a tendency to roll 
backwards when they come to rest. 
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involvi^s knowing what OUGHT to be occurring in a particular situation; this 
involves developing an individual and an institutional system of values to 
determine the oughts, and a system of management to correct errors and to keep 
the organi?.aLion on course. On the basis of a knowledge of the situation that 
exists and a value system that has been developed, a specification of what OUGHT 
to occur (or what outcomes are desired) is made. On the basis of what he thinks 
CAN happen, he tries something. On the basis of what DOES happen, he makes an 
adjustment and recycles. The direct implication of this assumption is that the 
administrator should develop .a reasonable means of determining desirable outcomes 
and a system of feedback, analysis, interpretation, and decision-making. 
Information Relating to the Assumptions 

What information is available to the administrator who makes these assumptions? 
It is recognized that the information cited is not exhaustive, but hopefully it will 
be beneficial at least in a suggestive sense (suggestive of the type of information 
that should be collected by the administrator, that is). 

Regarding dealing with what IS reality . Situations (including the organiza- 
tional environment) change over liiue, and Jiffcring siLuaLious have a differential 
impact on the behavior of teachers and the predispositions of students. Teachers 
behave differently in different situations, change over time, and their effec- 
tiveness is not very consistent (Brophy, 73) (exception: same material, dif- 
ferent students, short-term lesson; Rosenshine, 1970), Knowing this, and having 
the desire to know what reality is like at a given point in time makes the admin- 
istrator very aware of the need for precise and adequate sampling of data that 
might be available to him. Not only must he sample (or choose) what domains and 
variables he wants to measure (and this appears to be related directly to his cue- 
reduction ability); but he also must acquire the data from a given domain in such 
a manner that he gets a clear picture of this reality that he faces. I will not 
deal with this latter sampling question, but I would like to offer some suggestions 
of categories of information that should assist him in understanding this reality 
better. Your own analysis or subdivision of reality into component parts may lead 
you to different categories and measurement procedures, but these appear reasonable 
to ne at present. 

1. Environmental variables, including; sociological writings of current 

scene, personal observations of current scene, parental surveys, measures 
of organizational environment such as school climate, psychological 
reports of nature of learning. 
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2. PcrsonnliiU.ic or process variables: observation guides for rerording 
teacher behavior (particularly valuable if they are criterion referenced 
and low inference), observation guides for recording student behavior 
during learning activities, feedback from students of their observations 
of teachers, feedback from teachers of their observations of their own 
and student behavior. 

3. Outcomes or producf variables: achievement tests — commercial and teacher 
made, group referenced and criterion referenced; aptitude tests, affec- 
tive measures — dealing with interests, attitudes, temperament, prefer- 
ences, and expectations; and psychomotor tests involving skills devel- 
oped. 

It will be noted that a number of these variables do not deal with pupil 
growth measures. However, the theme of this paper implies that the administrator 
cannot deal with pupil growth measures in isolation from other variables with 
which he must contend; consequently, the need for the measurement of other 
variables in order to understand fully the reality with which he interacts. 

Regarding donling wf ^b what CAN occur . Although the administrator'^ ego 
tells him that teachers should behave very similarly to the way that he behaved 
in the classroom, and his high value for parsimony causes him to desire only a 
very limited number of ways of accomplishing goals, in the depth of his mind is 
a full realization that different processes can produce the same results. 
Research results would lead one to the same conclusion; we have many more "con- 
tributory" variables than either "necessary" or "sufficient" variables. 

And these contributory variables exist on different levels. For example, 
some research regarding student growth has been concerned with macro variables. 
In this regard, (a) schools do appear to make a difference, in that older students 
score better on achievement, (b) the socio-economic background of students is one 
of the better predictors of achievement, (c) learning is differential by subject 
matter and is sometimes sex linked, and (d) learning in some areas is related to 
the national and societal commitment to learning in those areas. (Feathers tone, 
1974) 

On more of a middle range of specificity, it appears that certain organiza- 
tional variables of schools (e.g., size of class or school, whether the school is 
"open" or graded) have very little impact on cognitive achievement. (Piatt, 1974) 

Many administrators are concerned with the implications of the macro and the 
middle range variables; however, there probably is greater interest in more 
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speciflc or micro variables. And it is here that wc have more extensive research 
results, some of which give the administrator relatively good cues to first 
estimates of what might be tried in the classroom. Recent reviews (Rosenshine, 
1971; Rosenshine and Furst, 1973) indicate that the following appear to have the 
most promise because of their relationship to pupil growth in achievement: 

1. Clarity of expression; very consistent positive correlation. 

2. Use of variety of instructional materials and procedures; positive 
correlation. 

3. Enthusiasm; significant and consistent positive relationship. 

4. Achievement-oriented and business-like manner; consistent positive 
trend. 

5. Teacher criticism of students; significant negative relationship between 
strong criticism and achievement, mixed trend vzith mild criticism with 
some positive relationships. 

6. Teacher indirectness and use of student ideas; consistent positive 
relationship, not all of which were significant. 

7. Opportunity to ]eam the criterion material; positive significant 
relationship. 

8. Use of structuring comments; large number of significant positive results. 

9. Use of higher level questions; significant differences in experimental 
groups (Gall, 1970) 

It is perhaps surprising and disconcerting to some that such variables as 
non-verbal approval, praise, warmth, l/D ratio, student talk, student participation 
have not been shown to be significantly and/or consistently related to pupil 
growth. Two comments may be in order here. First, these non-significant rela- 
tionships should serve as a warning to administrators not to mistake pupil 
happiness with instruction, the general popularity of a teacher, or busyness 
(active involvement) of students as true indications that achievement or changes 
of attitude are occurring. Second, since most of the correlational and exper- 
imental studies have been done over periods of one school year or less, we do 
not have much information regarding how these indicators relate to later learning 
and attitudes toward learning. For example, consider both short-term and long- 
term learning in both the cognitive and the affective domains; 

cognitive affective 

short-term 1 2 

long-term 3 4 
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Pupil yrowtl^ In cell 2 may promote a gain in cell 3 or 4, even though no 
gain occurs in cell 1. Likewise, a ^ain in cell 1 with negative results in cell 2 
may decrease pupil growth in both cell 3 and cell 4. Although considerable 
resources would be needed to test such a hypothesis on a large-scale basis, an 
administrator should be able to determine what results are obtained on a local 
level with reasonable use of resources • In effect, it seems important to me that 
administrators assist teachers in clarifying their hypotheses with relation to 
these cells and acquiring information about results. 

Some people express concern because many of the research results that we 
have are correlational rather than experimental, because we cannot , npoint the 
precise cause of learning and attribute specific results to the actions of a 
particular teacher. However, I would argue that school learning always occurs in 
a diseased environment rather than an antiseptic one; that, in fact, learning 
orcurs in any situation because of an interaction effect of teacher behavior and 
exLtaneous situational factors. Therefore, one is not so concerned with whether 
the teacher behavior in and of itself caused the results as the fact that the 
teacher behavior In conjunction with other variables in operation in the situation 
caused pupil growth to occur. This point of view emphasizes the idea that either 
experimental or correlational information which expresses relationships between 
procedures of teachers and pupil growth measures may be beneficial in estimating 
what initial procedures should be tried in specific local situations. 

Other information that appears to have relevance to the general theme of 
this paper, and" to relate to what CAN occur, is that within certain subject matter 
areas certain methods affect different subgroups differentially. (Berliner & 
Cahen, 1973) Although the research evidence regarding trait-treatment interaction 
is not as fruitful as one might hope in view of the emphasis on individualization 
of instruction that we have seen in recent years, it is sufficiently strong to 
support the view that administrators should assist teachers in formulating hunches 
regarding what is most likely to occur with different types of students and then 
checking to see if the different procedures are effective in the manner hypoth- 
esized. 

Regarding coping with manipulations . What information does the administrator 
have available that will allow him and the teacher to deal with what DID happen 
after certain procedures were tried? First, he has available all of the measurement 
capabilities that help him to describe the reality he faces. Measurement of these 
variables will help him to know relationships among process-product variables for 
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his specific situation. As he examines such measures, he will become very 
aware that certain information is of very little value to him and the teacher 
in making plans for the next cycle of activities. For example, it does little 
good to know grade placement scores or mean scores on criterion tests, ^^lat is 
beneficial is to know which desired outcomes were achieved and which were not — 
and by which subgroups of students. And, he also needs to know what specific 
teacher behaviors wore used. This information allows him to determine whether 
the procedures planned were actually implemented (procedures are often condemned 
for not acquiring desired results — v/hen the real problem ja that the procedures 
were not ^"lly in»;)iemented) , whether the results were satisfactory, and whether 
the proceti.i.es should be replicated or modified. The important consideration to 
the administrator is whether the feedback he helps to provide and analyze con- 
tributes to decisions which need to be made by the teacher. 

If it is decided that the procedures did not produce the desired results, 
then the administrator returns to the information base regarding what CAN occur to 
acquire what might be considered to be the next-best hunches regarding what pro- 
cedures to use. If the planning-collccting information-reviewing results cycle is 
relatively short, corrective action prevents serious errors from creeping into 
the instructional system. 

In addition, the administrator needs good analytical and decision tools to 
manage the information regarding processes used and results obtained. His basic 
analytical tools are those of subdivision of information into fine enough parts 
to be psychologically meaningful, and statistical techniques that will allow him 
to discuss clusterings of data. Where processes are found not to have been 
implemented as planned, he needs access to control mechanisms which improve that 
implementation. To utilize the information adequately, he needs communication 
skills that will facilitate acceptance of the feedback that has been analyzed and 
interpreted. 
Specific Implications 

What are the specific implications of (a) the general theme that adminis- 
trators and teachers should use research information as a first estimate of what 
is likely to result from given procedures in a specific situation, and (b) the 
assumptions that an administrator must be able to deal with the reality of the 
present, to plan for what can happen in the future, and to cope with manip- 
ulations of the teaching environment? 
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1. Generally > tcachors should bo sielected l:or their adaptability, their 
capabf'lity of solving problems that arise on the Job, their enthusi«ism 
for the task at hand, their ability to explain ideas clearly, and their 
task orientation and desire to produce results. These attributes 
appear to be less susceptible to training, yet do appear to have a 
reasonable linkago to pupil j^rowth in a variety of situations. 

2. Generally, preservice or inscrvice training should be provided to 
develop the information base, the attitude, and the skill to use a 
variety of instructional procedures and materials, to use criticism 
judiciously and selectively, to be indirect in approach to the extent of 
using comments and ideas of students, to structure lessons by providing 
advance organizers and lucid summaries, and to use feedback for modifying 
instructional procedures. 

3. Since inforr;ation available regarding pupil growth research is such that 
we do not have general and lawful relationships between teacher processes 
and pupil growth, administrators should not evaluate teachers on the 
basis of procedures alone. To treat process, e.g., the use of discovery 
procedures or multi-sensory approaches, as the most important criterion 
for evaluation (and subsequent supervision) is to ignore the impact of 
intervening variables in specific situations. If one looks only at the 
process side of the process-product relationship, he does not know what 
CAN happen — he merely knows the procedure used rather than whether or not 
it worked in the manner desired. A related implication is that to 
examine pupil growth without collecting information regarding procedures 
used prevents one from diagnosing difficulties or replicating processes. 

4. It takes time and skill to operate in the manner I have been describing. 
Therefore, the administrator should be trained to observe, analyze, and 
communicate. In addition, the administrator should be able to concep- 
tualize the distinction between evaluation and supervision — and should 
design an individualized supervision based on information from the 
evaluation process. This implies that administrators should do something 
that a large number of them do not do, viz . , integrate evaluation into 
their management or supervisory style. 
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Conclucl in}] (!oniinont.H 

Wliich brinji;,s us back to the baoic theme of the paper, viz , , that it makes 
sense for adinlnlntrators and teachers to adopt a stratepy that uses research 
findings regarding pupil growth as an initial estimate of what activities 
are likely to produce best results in a given situation — but currently should 
not consider the research results as genaral rules to follow. If this strategy is 
followed, administrators and teachers will begin to use observational systems 
to test hypotheses in local situations, ralher than simply to document desired 
behavior (which Rosenshine and Furst, 1973, indicate appears to be the more 
prevalent practice). Likewise, the strategy will have the potential durability 
for surviving over time, since it has a built-in capability for evaluating the 
local impact on pupil growth of research done in very limited situations — but 
advocated for general adoption prior to conducting research in normal classroom 
situations. 

The strategy appears to be compatible with the scientist and to use his 
iei>eaich results for making tirst estimates. It can use the creative talents 
of the artistic teacher who has insight into the optimum blend of activities for 
a particular group of pupils, and it requires the execution of well developed 
craftsman skills. The use of such a strategy may not make the administrator any 
less peculiar, but perhaps his peculiar peculiarities will be accepted and 
understood by both the research psychologist and the classroom teacher. 
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